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PREFACE 



The Research on Evaluation Program is a Northwest Regional Educational 
Laboratory project of research, development^ testings and training 
designed to create new evaluation methodologies for use In education. 
This document is one of a serleb of papers and reports produced by 
program sta££» visiting scholars^ adjunct scholars^ and project 
collaborators — all members of a cooperative network of colleagues 
working on the development of new methodologies.* 

What Is the nature of committee hearings as used In congressional 
Inquiry and policy formation? Can committee hearings be adapted for 
use In educational evaluation? This report discusses these and other 
questions In assessing the potential of conunlttee hearings as a tool 
In evaluation. Included In this report Is a sample application of a 
committee hearing procedure In evaluation and an extensive discussion 
of the various elements of the committee hearing process » Including 
committee operation, staffing, and procedural rules. 



Nick L. Smith, Editor 
Pfiper and Report Series 
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COMMITTEE HEARINGS AS AN EVALUATION FORMAT 



The Idea of using a committee hearing format for educational 
evaluation Is not necessarily unique — a blue ribbon pdnel approach 
to evaluation, for example, could be considered to be a relevant 
lineage; and In accreditation models the self study component often 
utilizes committees. Educational evaluators, however, may examine 
the use of committees In other environments to gain new Insights 
Into the potential of committees as an evaluation format. This 
paper Is undertaken In that spirit. In particular, Congressional 
Investigative hearings are used here as an analogy to Instruct 
educational evaluators. 

This paper includes three sections. The first section describes 
the concept of the investigative committee as practiced by the United 
States Congress; the second section reviews a recent implementation of 
a committee hearing format in educational evaluation; and the third 
section provides an outline of a committee hearing approach with 
comments directed at the application of the format. 

While the ultimate purpose of this review is to provide a 

^ An earllet version of this paper was presented at the American 
Educational Research Association annual meeting in San Francisco, 
California, in April 1979. The current document is a revision of that 
paper based on the perceptive comments of Dr. Cpnrad Katsenmeyer of 
the National Science Foundation. 



practical outline of. a committee hearing approach to educational 
evaluation^ It Is also hoped that practitioners can also use the 
paper to Identify features applicable to other approaches to 
evaluation. 

Congressional Hearings 

Since the advent of television^ the Image of Congressslonal 
hearings Is not as distant from the American public as It once was. « 
Television news coverage » however » often portrays the events 
surrounding the public phase of a Congressional Investigation^ and 
focuses attention on only a portion of what Is actually a broader 
effort. An analogy should consider the broader process which would 
Include some conceptualization of the parent body for the committee; 
the authorization for the committee; the composition of the committee; 
the activities of the committee Including Investigation^ hearings » 
reports » and subsequent action. 

In this broader perspective » educational evaluators will find 
many parallels between the committee approach and existing practices 
with which they can be comfortable. Their greatest discomfort, however, 
may arise In relation to the event of the hearing Itself.^ 

Parent Body 

Parliamentary and legislative bodies utilize committee systems 
• to accomplish many of the Important functions assigned to them. The 
United States Congress Is no exception. These legislative groups are 
the "parent bodies" for the committees. That Is, they provide a source 
of origin and legitimacy for the committee. 



^ A foreshadowing of this might be suggested by Popham and Carlson^ s 
(1977) reaction to the "deep dard deficits" of adversary hearings. 
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Congress Is not monolithic In philosophy and embodies divergent 
approaches to conceptualization and resolution of problems. The use 
of committees Is well proven In this politicized setting In gathering 
and testing Information against a variety of frames of reference. 

Authorization 

Investigations are Initiated when a member of the parent body 
Introduces a motion to extabllsh a select committee for Investigation. 
In addition to that consideration » the parent body will specify the 
charge to thft Investigative committee. The specification of the 
charge Is Important. Too narrow or too vague a specification would 
contribute to an Ineffective committee. Committees have been known 
to return to their parent body to obtain clarification of their 
authorization for Investigation. This Is considered to be a signal 
that the Initial authorisation was not adequately considered (Mbrrow» 
1969). 

The nature of the authorisation Is Important for a variety of 
reasons. Primarily It serves to focus the activities of the committee. 
It also relates to one of the attempts to Insure the fairness of 
committee Investigations. Fair play for witnesses » for example » 
Includes specifying the nature and scope of the Investigation. 

Committee Membership and Staffing - 

In the Congressional setting » the chairperson of the Investigative 
committee Is determined by party leadership with consideration for the 
Senator or Representative Introducing the resolution Into the parent 
body. The functions of the chairperson are critical to the liDplemen* 
tatlon of the committee. The chairperson hires staff; supervises staff; 
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establlshes schedules; manages proposed legislation on the floor 
of Congress; and serves on conference connolttees when legislative 
differences aria being worked out between the Senate and the House* 
If the chair does not undertake these functions directly (s)he 
often has the power to designate the person who will. (Diamond, 
1976; Morrow, 1969; Price; 1972) 

Committee membership Is also designated by the party leader- « 
ship. Select committee membership for the party of the chairperson 
Is often Influenced by the prospective chair. Depending on the 
Import of the Investigation, committee membership Is also based on 
reputation and stature In the parent body and the nation. The 
composition of the committee may take Into account the significance 
of the Issues to be Investigated, the expertise desirable, or the 
expressed Interest of CongressmeiL. In select committees It Is 
often the case, however, that members are faced with the necessity 
of becoming an Instant expert on the topic under Investigation. 

Two general types of staff serve the commit tee**-cler leal and 
professional. Committee organization may have a chief of staff 
to whom both groups are responsible, or a structure that utilizes 
a chief clerk and a chief of the professional staff vho report to 
the committee chairperson (Heaphey and Balutls, 1975; Kofmehl, 
1962). 

Clerical staff perform many of the support functions necessary 
for the operation of the committee. However, during the press of 
events, clerical staff have been used In professional -like capacities. 
For example, they have conducted Interviews, assessed written materials, 
and have written portions of reports. 

® If) 
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A chief counsel often heads the professional staff. The 
chief counsel Is typically selected by, and Is responsible to the 
chairperson of the committee. Ideally the chair and the chief 
counsel cooperate In all phases of the work of the committee. 
The chief counsel niay also be supported by a minority counsel 
assigned to assure Investigative scope In the Interests of the 
minority members on the comntlttee. The chief counsel Is to direct 
the Investigation, the data analysis, plan the public hearing 
actlvltes, brief the committee members, conduct a protlon of the 
questioning at the hearing, and direct the preparation of the 
committee report. 

Professional staff for committees represent a variety of 
backgrounds and skills* Some of the disciplinary backgrounds 
Include law, economics, political science, and Journalism. 
Professional staff are selected as best as possible for skills 
pertinent to the nature and scope of the Investigation. At times, 
however, they find themselves engaged In tasks for which they have 
little preparation. This is quite often true where staffing Is 
limited, and where the press of time requires tasks to be under- 
taken by whomever happens to be available. Onder such circumstances, 
technical support by the chief counsel or other professional staff 
may be used If possible^ Even committees staffed with over 30 
professionals* such as the Watergate Committee, may be forced to 
make assignments that are *'out of sync*' with the staff member^ s 
primary skills. (Heaphy and Balutls, 1975; Johnson, 1951; Huwa 
and Rosenthal, 1976; and Kofmehl, 1962.) 

It Is Important, therefore, to attempt to anticipate work 
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loads In planning cotomlttee Investigations. Further^ In technically 
complicated questions » such as the ballistics snd forensic medicine 
questions of the Assassination Committee Investigating the Kennedy* 
King murders » special studies may be commissioned by the committee to 
supplement staff expertise. 

In Congress » feuds between and among committee members as well as 
between and among committee members and the pro- * 
fesslonal staff are not unlcnown (Johnson^ 1951; Vaughn» 1972). The 
balance necessary for the Integrity of the committee will require 
careful mediation. A chairperson of an evaluative committee may 
not be allowed the extr^ne authority traditional In Congressional 
commlttees-^however even that Is changing (Davidson and 01eszek» 1977). 
The role of the professional staff In limiting the constricting effects 
of contention Is Important (Prlce» 1972). Shared direction or 
utilizing voting to support or deny proposals can be useful (Prlce» 
1972). 

Investigation Activities 

One of the first tasks of the committee and its counsel is to 
determine the rules for committee practice (Brcwa» 1975). The rules 
relating to hearings now include such considerations for fair play 
as Informing potential witnesses of the scope of the investigation 
and their right to counsel. Other rules for the commitee may 
relate to questioning and other procedures. 

Descriptions of the investigative activities of committees 
suggest that activities range from routine to dramatic (Bailey» 1950; 
Hamilton^ 1976; Johnson^ 1951; Schlessinger and Bums» 1975). The 
task has been described as putting a Jugsaw puzzle together. 



The tactics used In the Investigation phase of the work of 
the committee often reflect the methodologies familiar to the 
professional staff. Investigative methods as applied to criminal 
Investigation^ history^ Investigative journalism^ auditing^ economic 
analysis^ survey work» and cither specialties as may be suited to 
the task undertaken are often Implemented. 

The Investigation activities may even continue during the * 
course of public hearings. The Investigative Information Is used 
to provide background for the hearings In general » and to brief 
committee members regarding potential testimony of witnesses. 

The final stages of Investigation prior to the public hearing 
often reflects the lines of questioning to be Implemented In the 
hearing. A variety of positions have to be tested. 

Hearing Activities 

Preparation for the public hearing phase Includes setting out 
the schedule of events. This amounts to an orchestration of Inform 
matlon to be presented at the hearing. It Is case building as a manner 
of argumentation. For example^ Vaugha (1972) Indicated that the House 
Un-American Activities Committee format for the "Hollywood ten" began 
with friendly witnesses setting the stage » and then progressed to 
hostile witnesses. The strategy not only set the stage with informa- 
tion » but also dramatically orchestrated an impression (hat Communist 
infiltrators were pervasive in the movie industry. The intent was to 
elicit antipathy toward Coimnunlsts and Communist sympathizers. 

The chief counsel develops the initial agenda for the hearing. 
This is then shared with the committee chairperson and then brought 
to the committee as a whole. Once the plan is approved^ the chief 



counsel supported by the chief clerk» make arrangements to notify 
witnesses and Insure the implementation of the plan. Rooms , 
facilities, and equipment are secured. Personnel are added, In- 
cluding stenographers where necessary. Witnesses are notified and 
are provided with Information depicting the scope of the hearing, 
the committee rules, and the rights of witnesses. 

Questioning In the hearing Is undertaken both by staff counsels • 
and the committee members. Staff counsels are typically assigned the 
task of eliciting Information that provides setting, establishes tha 
parameters of the Information available through the witness » and 
obtains answers to questions about technical matters not In tha 
field of expertise of committee members. Committee members, supported 
by committee and personal staff » ask questions within the scope of 
the rules of the committee. The questioning strategies of committee 
members often reflect hypothesis testing. Each committee member may 
have a different perspective to explore, or In some cases, a party 
position may be used as a single perspective for framing questions. 
At those times, a caucus of party mamebers may be used to discuss 
and set strategy In advance. 

Once testimony has been taken, witnesses may be thanked for 
their participation. Finally, committee members may make statements 
highlighting their frame of reference vis a vis the testimony. 

CotTwn'f ttee Reports 

When the public and closed sessions of the hearing have been 
concluded, committee working sessions are scheduled to facilitate 
the preparation of a final report and recommmendatlons. The working 
sessions Identify the major points Identified, debate the support for 
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perspectives, establish the outline of the final report, Identify 
possible recommendations, suggest final timelines, and determine If 
there will be minority reports. Professional and clerical staff 
are allocated responsibilities for the production of the report and 
Its drafts. Additional committee sessions discuss drafts of the report 
and prepare strategy to present the report to the parent body. Finally, 
when the report Is finalized, a vote Is taken to transmit the report * 
to the parent body and the committee chairperson typically Is charged 
with managing the strategy to obtain legislative consideration. 

A Recent Implementation: Committee Hearing as an Evaluation Format 

Developmental Perspective 

A recent attempt to develop an adversarial hearing approach 
to evaluation was undertaken by Wolf (1974). In his work. Wolf 
suggested that an adversarial approach had potential utility to 
remedy many of the perceived Ills of educational evaluation. Wolf 
applied a Judical analogy to evaluation, and Implemented a full 
scale application of the Idea at Indiana University In 1975 (Amsteln, 
1975; Wolf, 1975). 

A major strength of an adversarial approach to evaluation Is 
the testing of data and Interpretations of data through examination 
(Stenzel, 1975ai Wolf, 1974). InWolf*s 1975 application, however, 
the panel sitting to hear the evidence also put questions to the 
witnesses. Stenzel (1975b) noted In review of that point that many 
evaluations had limited credibility In settings where a spectrum of 
contending Interests were Involved. For the State Educational Agency 
audience, for whom Stenzel ^^s writing, settings were often politicized-^ 
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parents v£ educators va^ advocacy groups; or fiscal liberals vs fiscal 
moderates vs^ fiscal conservatives. Persons holding contending value 
perspectives were not likely to accept the results of an evaluation 
that did not coincide with their Interests. 

Stenzel (1975a) » In meta-^evaltiatlon of the judicial format, 
suggested that there could be other approaches to adversarial 
evaluation. He later suggested that evaluative findings meta-eval-^ 
uatlvely tested from a variety of points of view would better serve 
as the basis for policy and action (Stenzel* 1975b). Indeed* a meta- 
evaluatlon utilizing a committee hearing format was proposed and 
Implemented by the Illinois Office of Education. 



The Application 

Under contract to the State Office of Education, an evaluation 
had been conducted by a third party evaluator on a set of thirteen 
regional projects which provided services for low prevalence handicapped 
children In Illinois. The third party evaltiator agreed to appear 
before a committee to present the final evaltiatlon report and to 
answer questions regarding the conduct and results of the evaluation 
(Illinois, 1975). 3 



The Investigative procedure used by the third party evaluator was 
based on a site visit to each of the thirteen regional projects by a 
team of experts. Prior to the visit, each project was to have submitted 
responses to a questionnaire designed to elicit general descriptive In-^ 
formation a^^ut the project. Questionnaire responses were to be reviewed 
by members of the site visit team prior to their visit. The site visit 
team consisted of a team leader; experts In the areas of vision, hearing, 
and orthopedlcally handicapped to review programmatic aspects of the 
project; an accountant to review the fiscal procedures; and, where possible, 
an expert In administration. If no special administrative person was on 
the team, the team leader conducted the Interviews to review the administrative 
practices of the project. As a standard for providing judgments about 
the op<^ratlon of programs for low prevalence handicapping conditions, 
three statements representing Ideal programs In vision, hearing, and 
orthopedlcally handicapping conditions were prepared. Data were reviewed 
against these standards and 12 major recommendations were provided for 
O the final report ( Quigley, 1975). 
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The committee established by the Illinois Office of Education 
to hear the testimony of the third party evaluators and to examine the 
strength of the findings through questioning was broadly based. The 
committee included three members of the Illinois Office of Education 
professional staff. This Included the .Deputy Superintendent for 
Program Services, who acted as chairperson for the committee. Non- 
office participants included three representatives of groups concerned* 
with the education of the handlca^jped, and representatives of the 
projects subjected to the evaluation. 

The third party evaluators, who were the witnesses providing 
testimony at the hearing, were advised by a "counsel" selected to 
provide prudent and thoughtful perspective If questioning were to 
touch upon matters of contract and obligation. Committee members were 
also advised by ^^counsel** selected to represent another Institution 
or group similar to that of the committee member. In this respect, 
the "counsels to the committee** were not technical advisors, but 
rather persons providing additional perspective from the particular 
constituency the committee members were to represent. 

The charge to the committee was threefold. First, It was 
charged with exploring the validity of the evaluation as It was 
conducted and as It was represented In the evaluation report. Second, 
It was charged with reviewing the recommendations presented In the 
report. Third, It was charged with the production of a written response 
to the evaluation report. 

The hearing began with prepared testimony presented by a number 
of the Investigators Involved In the third party evaluation. Once 
the prepared statements had been made, committee members were allowed 
an opporttmlty to question the witnesses. Committee members were 



allocated time for two rounds of questioning. The first round of 
questioning was to Initiate topics and to elicit major 
responses. The second rbund of questioning was not to be used 
to initiate new topics but rather was to elaborate or follow up 
on information previously Introduced. 

Once questioning was completed^ the third party evaluators 
were thanked and the hearing adjourned. Each of the ^committee 
members were charged with the task of producing a ^^mlnorlty" 
report. These reports were compiled Into a meta-^valuatlon report 
ClUlnols, 1975). 

Two gyJ^^^1-fnftt!^rtnj^ of the Committee Hearing 

Two seperate studies of the committee hearing were completed. 
One study was undertaken by one of the third party evaluation group 
(Jones » 1976); and the second study was a survey of participant 
reactions conducted by the Program Evaluation Unit of the Illinois 
Office of Education (1975). This section vlll summarize the results 
of each study. 

Jones (1976) provided a comprehensive review and critique of the 
hearing. The standard used In the critique reflected adversarial 
procedures as suggested by Wolf (1975) and Stenzel (1975). 

Jones (1976) Indicated that sufficient care had not been taken 
to delinlate Issues for the coimnittee and that as a result the 
efforts of some of the questioners In the hearing were misdirected. 
Some of the misdirected questions put to the evaluators » for example » 
should have been put to the Illinois Office of Education as commissioners 
of the evaluation. Jones also Indicated that the examination of the 
argument constructed by the third party evaluators was Ineffective. 



He Identified several features of the hearing contributing to this 
weakness. First was the limited time allowed the task; second was * 
that cross-examination was backing In the committee process; and 
third was full disclosure of all evidence was Impossible under the 
format used.^ 

Another review was also conducted by the Illinois Office of 
Education. Stenzel Cl977) reported on the perceived strengths and 
weaknesses of the process based on a brief questionnaire used to 
elicit comments from selected participants and observers. The 
strengths included three points. First, the committee process made 
the review of the evaluation a public activity, disallowing a "deep 
6" fate for the results. The second strength was the committee 
process served as a ^'trial 'baloon" for recommendations as reviewed by 
participating Interest groups. The third strength Included the 
meta-evaluative functions of checking for accuracy, checking for 
potential biases of evaluators, challenging methodologies, requesting 
clarification, and exploring implications of recommendations. The 
weaknesses reported by Stenzel Included ineffective questioning 
strategies by committee members; the great expenditure of time and 
effort by participants required to yield a product; and the lack of 
coherence of the final product. 

These two perspectives obviously can be useful in refining a 



One approach to be used in the hearing was to have included a prior 
list of questions submitted to the witnesses by the committee in order to 
allow preparation where technical data were required. This was not fully 
implemented due to restrictions in lead time. 
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a committee hearing approach to evaluation. They should be^ kept 
in mind, but also should be tempered by the nature of the referent 
of the analog* For example, although it is imperative that there 
needs to be sufficient time available to engage in a committee 
evaluation process as was suggested in these studies, the idea of 
examination and cross-examination from judicial evaluation is not 
justified as it is replaced by examination from the variety of • 
value perspectives represented by manbers of the committee. Further 
examination of this implementation will be incorporated as comments 
in the following section. The next section is provided as an out- 
line of components of a Congressional-li}:e connnittee hearing process. 



Ironically, as evaluators explore the potential of committee 
hearing formats. Congressmen are exploring alternatives to their 
own tradition. The ^ommittee approach has a history of misuse, 
and it is costly in time and money. Yet, as consumers of evaluation 
studies become more sophisticated, and as studies of heralded importance 
are shredded through peer review and public debate, it may be best 
to work with adversarial approaches which incorporate plurality of 
perspective. Committee hearings may help in this. There is much 
in the committee approach to evaluation to recommend it. 

This section is presented in outline format. The headings 
refer to the committee concept as was presented in the previous 
section. Additional comments are included which are intended to 
suggest the potential of the analogy for educational evaluation. 

I. Parent Body 




Some Potentials of the Committee Hearing Format 




Committees are often established by deliberative bodies to 
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perform a variety o£ tasks. In legislative bodies one o£ those tasks 

may be investigation. In evaluation settings » institutions or organizations 

may find that a coimnittee*styled evaluation could be compatible with 

its structure and decisionmaking process. School boards^ for example^ 

might consider a committee hearing approach as part of its process 

to provide informed policy statements to guide educational practice. « 

. 

A^. Authorization 

In establishing investigative committees » the parent body provides 
a charge to the committee which deliniates the task to be undertaken. 
In evaluation^ a commission to conduct an evaluation could be parallel 
to the Congressional authorisation for a committee. Specifications 
are often included in a request for porposal or in a contractual 
agreement for an evaluation. Such specifications provide focus for 
the evaluation. 

The meta-evaluative function of the Illinois hearing placed an 
emphasis on the examination of methodoology that does not appear to 
be a typical Congressional concern. The combined emphasis on 
methodology and outcomes in the Illinois hearings however does 
appear to be desirable » since one of the criticisms of the Congressional 
approach to investigation is the lack of care regarding generalizability. 

B^. Committee Membership 

Committee membership Is typically drawn form the parent body. 
To apply the committee analogy to an evaluation, however, may cause 
some hesitation among those who hold to the "impartial^fully-informed- 
observer" school of evaluation. Selecting members of the parent body 
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to be on a committee of investigation suggests that biases may invade 
scientific detachment. Certainly Congress struggles with the balance 
between representation of the interests of the nation and submission 
to special interests. The political process > it is argued, in part 
accounts for that through public debate and consideration. In educat^]^- 
al decision making bodies, the public nature of the hearings may also 
serve to assure that matters are adequately considered. 

ft 

In evaluation jargon, the "parent body" has a parallel in the 
idea of 'Mecision makers" or those who authorize ^ evaluation. Indeed, 
it may be that the decision makers themselves differ in philosophy 
from one another, or that they operate within a setting where contending 
frames of reference have to be taken into account when action is 
proposed. It may be that a committee hearing evaluation will have 
utility in such a setting. / 

For example, school boards or boards of education are not likely 
to be monolithic in philosophy. In fact, to the extent that their 
manbership reflects diverse constituencies they are politicized bodies* 
Traditional evaluations serving their decision making functions face 
reduced credibility in proportion to the lack of compatibility of the 
findings with the advocacy positions embodied in the group. 

On the other hand, a committee approach more compatible with the 
"inpartial-fiilly- informed-observer" position may be seen in the Presidential 
commission conducting the Three Mile Island inquiry or in the British 
Parliamentary Commission discussed by Rhodes (197S) . As those approaches 
are more akin to the blue ribbon panel approaches applied to evaluation, 
they will not be explored here. 
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II . Committee Operation 
A. Organisation 
1^. Chairperson 

The Committee chairperson in Congress is a powerful fugure in the 
operation p£ the committee. The role o£ a chairperson as a feature 
of an evaluation may have some parallel in settings utilizing a 
contract monitor. That parallel is not perfect. The contract monitor * 
often does not ha^ve the broad functions of a committee chairperson, 
nor the status of a Congressman in the parent body. However, the idea 
that a person internal to a parent body have major involvement in 
the organisation and conduct of an evaltiative inquiry is an interesting 
idea. With a committee and a chairperson involved, evaluative results 
will be difficult to conveniently ignore. 

2. Staffing 

Two types of staff support the Congressional, committees—professional 
and clerical. 

a^. Professional Staff 

In the Congressional setting professional staffs headed by a chief 
counsel and at times a minority counsel, are hired to conduct the 
investigation. At times their efforts are supplemented by expert 
investigations . 

An evaluator or evaluation group hired under such circumstances 
is placed in much more of an ©nployee relationship than the consultant 
arrangenent often impl^nented at this time. There may be some 
advantage in this. It is possible that the employer- contractor will 
be much more sensitive to the constraints of time, the nature of the 
subject, the skills of the professional staff evaluators, and the 
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limltatlons of the data. 

The potential relationship o£ an evaluator orevaluators to a 
committee-^ like component of a parent body as a chief counsel and 
minority counsel places this approach into the specie of adversarial 
evaluation. The service of the professional staff in exploring 
hypotheses and gathering data that are relevant to those hypotheses 
also suggest an adversarial categorization for this approach. The * 
committee approach should allow examination of more than the two 
positions accomodated in a judicial hearing (Bullock, 1976; Stenzel 
197S). Investigative activities for the committee should be under- 
taken with the spectrum of committee interest in mind. The plural- 
ism thus served could, however, become unwieldy if too many positions 
are incorporated in the committee. 

The professional staff of a comitiittee hearing evaluation should 
be hired with the specific investigation in mind. In addition, 
specialized studies could be contracted in order to deal with technical 
topics. 

If the functions of the chief counsel are assumed by an evaluator 
for a committee, the counsel would select other professional staff, 
designate special studies, direct the investigation, supervise the data - 
analysis, plan the hearing, question witnesses, and guide the preparation 
of the report, A significant function in addition to these is briefing 
the committee members prior to the hearing so that they too will be 
able to conduct questioning of the witnesses. 

b^. Clerical Staff 

In the Congressional setting, the clerical staff, headed by a 
chief clerk, provide the necessary functional support for the 
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professional staff. Differentiated staffing as is suggested by the 
Congressional committee is a^familiar- feature of many large scale 
evaluation efforts. 

3. Committee Rules 

In Congressional committees one of the first activities of the 
professional staff in conjunction with the committee is to establish 
rules. In evaluation, Bullock (1976) and Wolf (1974) suggested 
that rules should be thoroughly considered in adversarial settings. 
It would seem appropriate to review such legal rules as those 
relating to evidence and such committee rules as those dealing 
with fair play. In evaluations relevance and materiality may be 
suppl atiented by matters of generalizability. Pair play considerations 
certainly could include informing witnesses of the scope of the 
investigation and of the technical questions they might expect in 
advance of the hearing (Stenzel 1977), 

The public examination of witnesses could indeed create 
sensitive situations. For example, in Wolf's examination 
of the Division of Teacher Education at Indiana University, 
witnesses were called to testify about the Division who at the 
end of the year would no longer have a job at the University. If 
their testijnony were not flattering to the Division, one possible 
tactic of cross-examination would be to <^estion the witnesses 
regarding the circumstances of their termination (Bullock, 1976). 
Rules of fair play could be useful in such cases. In addition^ 
evaluative committee hearings might call for a closed session to 
conduct <^estioning reflecting on the credibility of the witness. 

In evaltiative committee hearings^ where committee members 
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are members of a parent body and where questioning could touch 
upon the competence of individuals, the fulfillment of contractual 
obligations, or other topics with legal ramifications the right 
of witnesses to legal counsel may be a necessary feature allowed 
under the rules. The ramifications of public or executive 
testimony with adjudicative potential has not been explored in the 
literature of evaluation. In the Congressional setting, rules may 
allow Immunity or threats of contempt of Congress as tools to 
obtain testimony. Neither of these features are currently developed 
in relation to evaluation. In time, these matters will have to be 
confronted by evaluation, and with the application of both judicial 
and committee hearing evaluation it may be sooner rather than later. 

£. Planning 

Depending on the authorization and the available talents of 
professional staff, the activities necessary to gather documentation 
and identify witnesses for the hearing will be assigned to professional 
staff or contracted to specialises for other investigators. As 
unanticipated matters are uncovered, planning and r^la^ing will take 
place. In evaluation, where advance planning is an ideal and v4iere 
plans are reviewed as part of a response to a request for proposal, 
greater flexibility may have to be allowed. 

B^. Investigation 
U Tactics 

The investigative tactics utilized by the professional staff of 
Congressional committees often reflects their training in political 
science, feconcMics, journalism, or law. For evaluation efforts, this 
suggests a multidisciplinary approach that may be broader than is 
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typically contemplated in education. The evaltiative potentials o£ 
the wide variety o£ investigative approaches incorporated into the 
committee effort is beyond the scope of this singular effort. Each 
of the tactics common in Congressional investigations not currently 
in use or being considered for use in evaluation could be explored 
in seperate documents. What will be suggested here are the features 
of investigative evaluation. 

In a pluralistic settingi valuing and decision making include 
a variety of pranises and may be supported by different evidence 
or different weightings of evidence. Investigations for committees 
will have to be conducted with this characteristic in mind. An 
early task for the investigator is to become sensitive to the 
perspectives represented by members of the committee. Congressional 
staffers > for example, indicate that it is their task to anticipate 
the interests and needs of Congressmen before they are voiced (Price, 
1972). This effort could consist of determining the interpretive 
frames of reference, the hunches, the tentative explanations, the 
hypotheses, the types of evidence that are persuasive to committee 
menbers. Such considerations will have to be taken into account 
if the data generated through investigation and presented at a 
hearing are to be considered to be useful. In fact, the best 
available evidence should be sought in support of each of the perspec 
tives . 

2^. Committee Preparation 

The investigative briefings and summary of materials provided to 
the Congressional committee by their professional staff serve as a 
foundation for the public phase of the process—the hearing. In 
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evaluation^ preliiAinatry reports and oral presentations serve in a 
sijnilar manner. 

Meetings between the professional staff and the committee manbers 
should take place prior to the hearing to brief the committee manbers 
on progress, to discuss final selection of witnesses, to establish an 
agenda for the hearings, to plan the se<^ence of testijnony, and to 
deal with any anticipated problems. These meetings should culminate 
in a well conceived public presentation of testimony about data; and 
an ade<^ate, well informed examination of that testijnony by committee 
members.* Any final gaps in coverage of the issues under consideration 
should be filled, if at all possible, by further investigation and 
location of additional witnesses if necessary. 

As opposed to a judicial hearing nt^ere it becomes manditory to 
introduce all evidence into the hearing through appropriate witnesses, 
the committee hearing may review documents, depositions, or historical 
conditions without public examination of testijnony. In the judicial 
hearing, because of the plaint iff -defendant dichotomy, full public 
disclosure is necessary. In the committee hearings selection of a 
portion of the relevant evidence does not preclude consideration of 
other information. However, as committee m^bers are briefed or review 
summaries of documents, additional witnesses may be called if it is 
agreed that interrogation would provide necessary illumination, or that 
public opinion would require inclusion in the hearing. 

Hearings 
1_. Functions 

Pact finding and public activity are major functions of Congressional 
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hearings. In addition^ committee m^bers are able to test their 
personal frames of reference against the data. In evaluation^ this 
is possible in the judical approach to evaluation* The difference 
allowed in a committee format is the accomodation of a variety of 
questioning perspectives* 

In order to obtain some perspective how this can apply to evaluation, 
again it may be instructive to review the practice of judical evaluation* 
Bullock's (1976) account of an adversarial hearing at Indiana University 
suggests at least two approaches to case building* The first 
approach is a "discovery" approach in which the investigation is used 
to determine what data are available and then to derive the best 
possible argument from that information* The second approach is to 
outline the most plausible argument and then to gather data around 
that framework* In the first case the orchestration of the hearing 
is left until late in the preparations; in the latter case> the general 
outlines are apparent from early in the effort. 

Questioning tactics are a matter of concern in adversarial eval- 
uation* As managers of organizations or members of school boards are 
not necessarily experienced in the art of interrogation, that is a 
potential weakness of a committee hearing evaluation process* It was 
noted by Stenzel (1977) as a difficulty in the application of the 
committee hearing by the Illinois Office of Education* In addition, 
other factors may contribute to ineffective cpiestionpjig* These may 
include a lack of technical expertise in the matters under consider- 
ation, an inappropriate proportioning of effort, and the negative 
effects of personal biases* 
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The role o£ the evaluator as chief counsel, therefore, is 
important. It is critical that the evaluator possess and utilize 
questioning skills to disclose the parameters of the potential 
testimofny of witnesses, and to identify the salient points in that 
testimony* Bullock il976) suggested that good questioning in a 
judicial hearing should effectively use yes or no questions, ask 
one question at a time avoiding questions within questions, break 
questions into component parts, and phrase questions briefly and 
simply. Reflecting Wolf's dissertation (1974), Bullock summarized 
functions of cross-examination* Those functions could serve as an 
additional guide for questioning: retraction or contradition of 
testimony, disclosure of unfavorable el^ents in the case suggested, 
suggesting possible alternative inferences from the evidence presented, 
disclosing elements favorable to another position, and discrediting 
witnesses* The fact finding features of committee examination 
include some of these functions- It should be noted, however, that 
the abuse of some of these functions has led to recent Congressional 
reform. The mpst prominent of these reforms include fair play ideas. 

The committee hearing process has been criticized for conducting 
a trial by attenpting to create the impression of guilt or wrong doing; 

creating interventions prior to trials thereby prejudicing the 
outcomes of trials; and not allowing witnesses the opportunity to 
defend themselves* With such abuses in mind, however, the role 
of "tough" questioning can be suggested as testing the strength 
of the evidence or the opinions proffered by witnesses; looking 
for weaknesses in the positions explored by other committee members; 
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suggesting possible alternative inferences; and disclosing favorable 
elements to ones own line of questioning, 

2^. Types of Cominittee Sessions 

Congress utilizes both closed and public sessions. Public sessions 
take testimony and probe the implications of the data. Closed sessions 
are Used to explore sensitive issues. Hostile witnesses are often In- 
terrogated in closed sessions prior to public testimony. Evaluations 
too may have use for sessions of these types. 

The problem of how to deal with such an activity as interrogating 
a hostile witness may be dealt with in a closed session. Personnel 
matters obviously are closed session activities. The committee' 
rules should set parameters for the use of closed sessi6ns. 

D. Reporting 

1. Vforking Sessions 

Upon completion of the public phase of the committee work, 
working sessions begin to allow the committee manbers to deliberate 
upon the evidence, sketch out the contents of the report, and 
identify major findings. This working relationship between craranittee 
and professional staff may approximate a formative evaliiation setting, 
the incorporation of decision makers at this stage of events may go 
a long way to insure the use of findings and recommendations. 

2. Reports 

Vfritten reports summarize the major information obtained 
through all of the committee efforts— not just the hearing. 
Reports may include majority and minority Taports. Evaluation 
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reports do not often represent interpretations o£ the data in a 
majority-minority format. The closest approach to depicting a 
spectrum of possibilities may derive from adversarial evaluation 
reports or responsive evaluations. 

The inclusion of minority reports in evaluation work is 
not a common occurrence^ but appears to be a worthwhile 
consideration. This may incorporate some of the secondary 
analysis suggested by Cook (1974). The price paid for nniltiple 
reports, is reduced certainty for decision makers. 

E^. Follow Up^ 

In Congressional settings, bringing the results of committee 
investigations to the attention of other Congressmen and proposing 
legislation is appropriate, and is an expected outcome of committee 
work. In evaluation, with committee manbers involved in the evaluative 
process, such an expectation may also become a reality. 
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